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Abstract 

We study a stochastic network that consists of a set of servers processing multiple classes 
of jobs. Each class of jobs requires a concurrent occupancy of several servers while being 
processed, and each server is shared among the job classes in a head-of-the-line processor- 
sharing mechanism. The allocation of the service capacities is a real-time control mechanism: 
in each network state, the control is the solution to an optimization problem that maximizes 
a general utility function. Whereas this resource control optimizes in a "greedy" fashion, with 
respect to each state, we establish its asymptotic optimality in terms of (a) deriving the fluid 
and diffusion limits of the network under this control, and (b) identifying a cost function that is 
minimized in the diffusion limit, along with a characterization of the so-called fixed point state 
of the network. 
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1 Introduction 



We study a class of stochastic networks with concurrent occupancy of resources, which, in turn, are 
shared among jobs. For example, streaming a video on the Internet requires bandwidth from all 
the links that connect the source of the video to its destination; and the capacity (bandwidth) of 
each link is shared, according to a pre-specified protocol, among all source-destination connections 
that involve this link (including this video streaming) . A second example is a multi-leg flight on an 
airline reservation system: to book the flight, seats on all legs must be committed simultaneously. 
Yet another example is a make-to-order (or, assemble-to-order) system: the arrival of an order will 
trigger the simultaneous production of all the components that are required to configure the order. 

The engineering design of Internet protocols is a center piece of the bandwidth-sharing networks, 
the first example above. A key issue here is the real-time allocation of each link's capacity to each 
job class, which takes the form of solving an optimization problem for each network state. This real- 
time allocation scheme makes it very difficult if not intractable to evaluate the system performance 
over any extended period as opposed to real time. For instance, if the network is modeled as 
a continuous-time Markov chain, then the transition rate from each state is itself a solution to 
an optimization problem. Consequently, if one wants to optimize the system performance over a 
relatively long planning horizon, it is not clear what the resource allocation scheme one should use. 
In other words, there is a gap between real-time resource control and performance optimization in 
such networks. 

The objective of this paper is two fold. First, we want to overcome the intractability of perfor- 
mance evaluation under dynamic resource control by developing fluid and diffusion models: through 
suitable scaling of time and space, we show that the processes of interest in the network, such as 
queue lengths and workloads, converge to deterministic functions or reflected Brownian motions, 
under a broad class of utility-maximizing resource control schemes. 

Our other objective is to establish the connection between the real-time resource allocation 
scheme and its impact on the system performance. We show that while the utility-maximizing 
control optimizes in a "greedy" fashion, with respect to each state of the network, it will, in fact, 
minimize a certain cost function of performance measures such as queue lengths and workloads in 
the limiting regimes of the network. In this sense, the utility-maximizing control is asymptotically 
optimal. 

The fluid and diffusion limits are useful in characterizing the network performance for a very 
broad and important class of resource-control policies, whereas other analytical approaches appear 
to be intractable. Furthermore, the precise relationship between the utility function (which the 
resource control maximizes in each network state) and the cost function (which is minimized in the 
limiting regime) provides a useful means in deciding what type of controls (or protocols) to use so as 
to achieve certain cost-related performance objectives. Another advantage of these results is their 
generality, for instance, in requiring only very mild conditions on arrival and service mechanisms. 

The remainder of the paper is organized as follows. We start with an overview of the related 
literature in the rest of this introductory section. In Section we present details of the network 
model and the utility-maximizing allocation scheme. We also bring out a cost minimization prob- 
lem, and demonstrate its close connection with the utility maximization problem. In Section we 
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study the limiting behavior, in terms of fluid scaling, of the network under the utility-maximizing 
resource control, and establish the important property of uniform attraction: the fluid limit will 
converge uniformly over time to the so-called fixed-point state, which is a solution to the cost min- 
imization problem. In Section^ we consider diffusion scaling and prove the main result, Theorem 
IH1 the diffusion limit of the workload is a reflected Brownian motion, and the state (queue-length) 
process converges to a fixed point that is a minimizer of the cost objective; furthermore, both the 
workload and the cost objective are minimized under the utility maximizing control. Theorem |H] 
requires a key condition, that there is a single bottleneck link/server in the network. (A bottleneck 
link is one whose capacity is saturated by the offered traffic load.) We justify this condition by 
showing that it is equivalent to the so-called resource pooling condition, which is required in many 
models of heavy-traffic limit. 

Literature Review 

Circuit-switched technology, or, long-distance telephony, predates the Internet protocol. The cor- 
responding stochastic network model is often referred to as the "loss network" — calls that arrive 
when all circuits are occupied will be blocked and lost. Under Markovian assumptions, the loss 
network in steady state has a product-form distribution, which, however, is still computationally 
intractable due to the combinatorial explosion of the state space; refer to Ross |29| . 

Whitt's pioneering work [33] called attention to concurrent resource occupancy. It was moti- 
vated by studying the blocking phenomenon in loss networks. Kelly |161ll7j developed a fixed-point 
approximation for the blocking probabilities in loss networks (i.e., the blocking probabilities are 
solutions to a nonlinear system of equations), and studied the properties of the fixed-point mapping. 

As technology evolves, research in loss networks has moved into a new area, so-called bandwidth- 
sharing networks, where the service capacity (bandwidth) on each link is shared at any time among 
all the jobs in process at the link. These networks not only model existing protocols on the 
Internet (e.g., TCP), but also lead to studies of new bandwidth allocation schemes. Most of these 
real-time allocation schemes try to enforce some notion of fairness. Examples include the max-min 
fair allocation of Bertsekas and Gallager PQ, and its extensions to proportional fairness and other 
related notions in Fayolle et al. Kelly |18[ll9j. Kelly et al. [2^) Low [21], Massoulie and Roberts 
|27j . Mo and Walrand [2S], Wen and Arcak |32| . and Ye |35| among many others. Typically, the 
real-time capacity allocation takes the form of a solution to an optimization problem, with the 
objective being a utility function (of the state and the allocation), and the constraints enforcing 
the capacity limit on the links. Even under Markovian assumptions, this results in a continuous- 
time Markov chain with state-dependent transition rates that are the allocation schemes resulted 
from solving the above optimization problem. There is no analytically tractable way to capture 
the probabilistic behavior of such a Markov chain, such as its steady-state distribution and related 
performance measures. 

This has motivated research on fluid models of such networks, where the discreteness of jobs 
and randomness of their arrival and service mechanisms are transformed, via law-of-large-numbers 
scaling, into continuous flows traversing the network in a purely deterministic manner. Refer to for 
example Bonald and Massoulie 0, de Veciana et al. ^U], Kelly and Williams Massoulie and 
Roberts [2U, and Ye et al. |36| . Fluid models turn out to be particularly effective in studying the 
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stability of bandwidth-sharing networks under various capacity allocation schemes. (This is notably 
an extension of earlier studies on the stability of traditional queueing networks under various service 
disciplines. Refer to, e.g., Dai [S].) For instance, it is shown in |36| that under the usual traffic 
condition (i.e., each link has enough capacity to process all the traffic that uses the link), many 
"fair" allocations and related utility-maximizing schemes result in a stable network, whereas certain 
priority and (instantaneous) throughput-maximizing schemes may lead to instability. 

The fluid model in [22] provides an important first step towards establishing asymptotic regimes. 
Focusing on a "critical" fluid model (meaning, at least one link's capacity is saturated, i.e., equal to 
the offered traffic load), the paper introduces the notion of an invariant state — if the fluid model 
starts in such a state it will always stay in that state — and shows that under the law-of-large- 
numbers (fluid) scaling, the bandwidth-sharing network operating under a general proportional-fair 
allocation scheme converges to a fluid limit (which, quite naturally, evolves over time to an invariant 
state) . 

In the broader framework of processing networks of Harrison |121 ll'jj , concurrent resource oc- 
cupancy is one of the highlighted features, and the main approaches are built around Brownian 
network models. Our study reported here is mostly motivated by the recent works of Mandelbaum 
and Stolyar 25 a and Stolyar [HOll^- In [2Hj, multi-class jobs are processed by a set of "flexible 
severs" , meaning each server can process any job class albeit with different efficiency. It is shown 
that the so-called generalized cfi-rule — serving jobs according to a priority scheme that is deter- 
mined by the product of two factors: cost and service rate — is asymptotically optimal under a 
certain heavy-traffic regime. Similar kinds of asymptotic optimality results are established in 
for a "max- weight" resource pooling scheme in a general switch (e.g., including the crossbar switch 
as a special case); and in [3^, for a gradient scheduling algorithm (which is more general than the 
proportional fair protocol), applied also to a general switch. 

2 Utility Maximization and Cost Minimization 

2.1 The Network Model 

Consider a network that consists of a set of links, denoted C. There is also a set of routes, denoted 
1Z. Each route r G 1Z is a subset of links. Denote t £ r if link I is part of route r. 

On each route, jobs arrive at the network following a renewal process, independent of job arrivals 
on all other routes. Each job requires the simultaneous occupancy of all links I € r, for a period 
of service (or "connection") time that is independent of all other jobs. For each route r, denote 
the interarrival times between consecutive jobs as u r (i), and denote the amount of work (service 
requirement) each job brings to the network as v r (i), i = 1,2, . . .. Assume that u r (i) (i > 2) are 
i.i.d. random variables with mean 1/A r and variance a^, and that v r (i) are i.i.d. random variables 
with mean v r and variance b%. Denote the offered traffic load as p := ( / o r ) rg 7j, with 

Assume A r > and v r > (hence, p r > 0) for all r £ 1Z. 

An alternative view of the above network is depicted in Figure ^ there is a set of servers, 
corresponding to the links C; and a set of job classes, corresponding to the routes 1Z. To be 
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processed, each class-r job requires the simultaneous occupancy of all the servers I £ r. Figure 
Q shows an example with four job classes and three servers, with each of the first three classes 
requiring two concurrent servers, while the fourth class requires a single server. 




Figure 1: A network example 

A typical state of the network is denoted n := (n r ) r( zn, where n r denotes the total number of 
class-r jobs that are present in the network. Each server I G C has a given capacity, C£, which 
is shared among all job classes r 3 I. More precisely, one job (if any) from each class r 3 I is 
processed at any time, while other jobs in the same class waiting in a buffer. 

The allocation of the service capacities takes place in each state, denoted A(n) := (A r (n)) re -£, 
where A r (n) is the capacity allocated to class r when the network state is n. The actual time 
needed to complete a job then depends on its service requirement and the capacity allocated to it. 
Specifically, for the i-th class-r job mentioned above, provided it is being processed in state n, then 
the amount of work v r (i) associated with it is depleted at rate A r (n). 

Let M denote the set of all feasible allocations. Then, clearly (omitting the argument n from 

A), 



M 



{A : J2 Ar - Q ' W G £; Ar - °> Vr E n }- 



(1) 



r3l 



Throughout the paper, we shall assume that p G M, i.e., there is enough capacity in the network 
to handle the offered load. This is nothing more than the usual traffic condition, often a necessary 
condition for stability. 



2.2 The Utility-Maximizing Allocation 

The utility-maximizing allocation refers to the solution to the following optimization problem: 

max ) U T (n r ,A r ), (2) 
Asm 
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where M, the set of all feasible allocations, follows the specifications in (|T]l; and U r (n r , A r ), r £ TZ, 
are utility functions defined on the two-dimension nonnegative orthant = {x £ 3? 2 : x\, X2 > 0}. 
It is standard to assume (e.g., [SHI) that the utility functions are second-order differentiable and 
satisfy the following conditions: 

U r (0,A r ) =0; (3) 

U r (n r , A r ) is strictly increasing and concave in A r , for n r > 0; (4) 
diU r {n r , A r ) in strictly increasing in n r > 0, 

with 9 2 ^r(0, A r ) = and lim d^U r {n r ^ A r ) = +oo, for A r > 0. (5) 

Here the operator d^f is a shorthand for the partial derivative of function /(•) with respect to its 
k-th variable. Furthermore, we need another condition: An allocation is said to satisfy the partial 
radial homogeneity (or radial homogeneity for short) if for any scalar a > 0, we have 

A r (an) = A r (n), for any r S 1Z with n r > 0. (6) 

(Note, the qualifier, "partial", alludes to the fact that the above is only required for each r with 
n r > 0.) 

Listed below are some examples of the utility function widely used in modeling internet proto- 
cals: 

U r (n r ,A r ) = (3 r n r log(A r ), 
U r (n r ,A r ) = ~0r~^-, and 
A 1_a 

U r (n r ,A r ) =p r n°- J1 — ; 

1 — a 

where a > and /3 r > are given parameters. To motivate, consider the first utility function. The 
optimal solution takes the following form (from the first-order optimality equation): 

(3 r n r 

A r = , r£t, 

m 

where r\i is the shadow price (Lagrangian multiplier) associated with link t. That is, the optimal 
allocation among job classes is proportional to the number of jobs present in the network from each 
class. (Hence, this utility function is called "proportionally fair".) Note that the second utility 
function is a special case of the third one when a. = 2; and so is the first one, in the sense that 
its maximizer coincides with the maximizer of the third utility function when a — > 1 (refer to |2|). 
Also note that all three utility functions satisfy the conditions in (jHJ)-©; and the corresponding 
optimal allocations also satisfy the condition in @. 

Remarks. The familiar c^u-rule can also be represented as a utility maximizing allocation of a 
single server's (unit) capacity among a set of job classes, with a linear objective: 



^^C/ r (n r ,A r ) = ^^C r /i r l(n r > 0)A r 
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and a single constraint ^ r A r < 1. Hence, the solution is to let A r = 1 for the class r (with at 
least one job present) that corresponds to the largest C r /u r value. In the case of the generalized 
c/1-rule where the linear cost C r n r is replaced by a general function C r (n r ) (see, e.g., (23130]), the 
allocation is the solution to the utility-maximizing problem max^ r C^(n r )/u r A r . The utility is then 
U r (n r , A r ) = C' r {n r )iJ, r K r . Note, however, in both cases the utility functions are linear in A r ; hence, 
they do not satisfy the strict concavity condition in Furthermore, the strict increasingness 
(in n r ) of &2U in (j5J) implies strict convexity of the cost function C r {n r ), and hence cannot be 
satisfied by the c//-rule. Consequently, it has been known ( 36 ) that the plain c//-rule may not even 
guarantee stability, let alone asymptotic optimality, in the kind of networks that we consider here. 
On the other hand, for the generalized c//-rule, provided the cost function C r (n r ) is strictly convex 
and increasing, with C r (0) = 0, so that the condition in © is satisfied, then the strict concavity (in 
A r ) of the utility function, the requirement in (J1J), can be relaxed to non-strict concavity, and our 
main results below will still hold. The proofs, however, will involve more tedious, but non-essential, 
technical modifications. For instance, the solution to the utility maximization problem in @ may 
not be unique; and hence, the allocation A(n) can be any one of the optimal solutions. The radial 
homogeneity in (jHJ) needs to be modified accordingly. The proofs of the uniform attraction theorem 
and Lemma El need to be modified as well, since Lemma ^ no longer holds. 

Returning to the utility maximization problem in ((2} , we have the following optimality equations 
(which are both necessary and sufficient, due to the concavity of the objective function as assumed 
in ©): 

d 2 U r (n T ,A r ) = ^Tjt, A r >0,rGft; %-(^A r -c £ ) = 0, £ G C; (7) 

where r]t > 0, £ G £, are the Lagrangian multipliers. 

Furthermore, the following (partial) continuity property of the utility-maximizing allocation 
A(n), is known. It will be used in the proofs of the uniform attraction theorem and Lemma El 
below. 

Lemma 1 ( 36 ) Suppose a sequence of states {n- 7 , j = 1, 2, • • •} converges to n as j — > oo. Then, 
the solution to the utility maximization problem in (j2J) converges accordingly: 

A r (n J ) — > A r (n), as j — > oo, 

for each r G TZ such that n r > 0. 

2.3 A Cost Minimization Problem 

For each r G 1Z, associated with the utility function U r , we define a cost function C r as follows: 

C r (n r , A r ) = v r I d2U r (q, A r )dq; (8) 
Jo 

or, in differentiation form, 

d\C r {n r , k r ) = v r d2U r (n r , A r ). 
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(Recall, the utility functions are defined on 5R+; hence, so are the cost functions.) 
For example, in the case of the third utility function above, we have 

For the generalized c/x-rule, where we have U r (n r ,A r ) = C' r (n r )p r A r , the relation in (jSJ) also holds. 

We say that the heavy-traffic condition holds, if the offered-load vector p is a maximal element 
of M, i.e., p makes a non-empty subset of constraints in M binding: 



p r = ci, t6£*; Y,t>' <c '< l ? C "'- < 9 > 



E 

rBi rBl 



for some C* C C, and C* 7^ 0. Intuitively, we shall refer to each link in C* as a bottleneck link. 

Under the heavy-traffic condition, in particular, given the set of bottleneck links £*, along with 
a given set of parameters W£ > 0, £ G £*, consider the following optimization problem: 



mm 



^C r (n r ,A r ) (10) 

rtll 

s.t. v r n r > W£, £ G £*, 

n r > 0, r€?Z. 

That is, for any given allocation A, we want to identify a state n under which the total cost over all 
routes is minimized and the (average) workload at each bottleneck link I € C* is set to be greater 
than or equal to the required level W£. 

Let 1Z* denote the set of routes r such that r 3 £ for some £ G C* , i.e., each route in 1Z* involves 
at least one bottleneck link. Then, clearly, for any route r £ 1Z* , we must set n r = in the optimal 
solution to the minimization problem in (|10|) . since the cost C r is increasing in n r , following 
and (jl}, along with (|HJ). The remaining components of the optimal solution, (n r , r G TZ*), can be 
obtained from the following equations: 

diC r (n r , A r ) = v r ^2 Qi> n r > 0, r G 1Z*; &i(^2 v r n r - w e ) = 0, £ G £*; (11) 
where 0e, £ G £*, are the Lagrangian multipliers. 

Proposition 2 Suppose the heavy-traffic condition in Q holds; and suppose 1Z* = 7Z, i.e., every 
route involves at least one bottleneck link. 

(i) If n* is the optimal solution to the cost minimization problem in (|l()j) . with (A r = p r )rell i n 
the cost function; then, (A* = p r )r&v, must be the optimal solution to the utility maximization 
problem in © with n = n* in the utility function. 

(ii) Conversely, if (A* = p r )reTZ is the optimal solution to the utility maximization problem; 
then, the optimal solution to the cost minimization problem, with (A r = A*) rg 7^. in the cost 
function, must be n* = n, with n being the state vector in the utility function, and with 
we = J2r3E v r n r for all £ G £*. 



S 



When TZ* C TZ, both statements above still hold, with the correspondence between the two optimal 
solutions, A* = p r and n* = n r , restricted to r G TZ* . 

Proof. First, consider TZ* = TZ. Suppose the condition in (i) is true. From the heavy-traffic 
condition in Q, we know (A r = p r )r&TZ is a feasible solution to the utility maximization problem. 
To argue it is also optimal, note that for each r G TZ, we have, 

d 2 U r (n*,p r ) = u^diCrin^pr) = y~]gi, 

where the first equation is due to Q, and the second one follows from since n* is the optimal 
solution to the cost minimization problem. Hence, letting rj£ = 0£ for i G C* , and ne = for t £*, 
will satisfy the optimality equations in (J7J). 

To go the other way, follow the same argument. In particular, the first set of equations in 
(J7J) translates into the first set of equations in (|TT|) . with 0£ = rjg, for t G C* . The second set of 
equations in holds automatically. 

When TZ* C TZ, the above arguments will only apply to r G TZ* . Specifically, by setting r\g = On 
for i G C* , we can conclude that (A* = p r )r&n* is (part of) the optimal allocation to the utility 
maximization problem. (For r g" TZ* , A r is not specified; and neither is % for I £*.) Similarly, for 
the cost minimization problem, we have n* = n r for r £ TZ*. For r TZ*, as appointed out earlier, 
it is optimal to have n r = 0. □ 

In the literature, the cost minimizer n* (of the cost minimization problem with A replaced by 
p) is often referred to as a fixed point ([2HHS01) or an invariant state ((22]), provided the utility- 
maximizing allocation in that state is (A* = /9 r ) rg 7j*. We denote the fixed point corresponding 
to the workload w as n*(w). For convenience, we will suppress the second variable from the cost 
function to write C r (n r ) for C r (n r ,p r ) and C' r {n r ) for d\C r {n r , p r ), when A r = p r . 

To close this subsection, we present the following lemma on the continuity of the the fixed point 
n*(w), which will be used in the proof of the main results (Theorems H3 and |HJ) later. 

Lemma 3 The fixed point n*(w) as a function of the workload w = (wg)e^c* is continuous in w. 

Proof. It suffices to show that for any sequence of workloads (vectors) {w^\j = 1, 2, ...} satisfying 

W U) _> u ,(°) ) n*(w^) — > n' , as j — > oo, 

we have n' = n*(w^). 

First, the optimal value of the cost minimization problem in (fTU|) . J2 r ell Cr( n *( w ))> as a function 
of the workload w, is continuous in w. Therefore, we have 

J2Cr(K(w^)) -> Y, C r( n r(w (0) )) as j — > OO. 

On the other hand, the cost function C r {n r ) is continuous in the state n r , r G TZ; and hence, 

C r (n*(u> ^)) — > C r (n^.) as j — > oo. 
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Therefore, 

]Ta(<(^°))) = ^a(<). (12) 

Next, for any real number 5 > 0, let S" 5 denote the set of all states satisfying the following con- 
straints, 

n r > 0, r € 71; v r n r > —5, £ € C*. 

Then, it is directly verified that we have n' 6 S s . Therefore, we have 

n' € lim S 5 = Pi s>0 S S . 
8— >0 

In other words, the state n' satisfies the following constraints, 

Finally, the above constraints, combined with the identity (|12|) . imply that the state n' is also 
the optimal solution to the cost minimization problem in (|1U(I with w replaced by vj(°\ Since the 
optimal solution is unique, we must have n' = n*(w^). □ 

3 Fluid Limit and Uniform Attraction 

Here we start with presenting the main processes associated with the stochastic network introduced 
in the last section. First, the two primitive processes are the renewal (counting) processes associated 
with the job arrivals and the work (service requirements) they bring into the network: E(t) = 
(E r (t)) r( zK and S{y) = (5 , r (y)) rg ^, where 

E r (t) := max{i : u r (l) + • • • + u r (i) < t}, (13) 
S r (y) := max{i : v r (l) H h v r (i) < y}. (14) 

The two derived processes that characterize, along with the two primitive processes, the dynamics 
of the stochastic network are the queue-length process and the capacity allocation process: N(t) = 
(N r (t)) re n and D(t) = (D r (t)) r€ n, where 

N r (t) := N r (0)+E r (t)-S r (D r (t)), (15) 

D r (t) := [ A r (N(s))l {Nr{s)>0} ds. (16) 
Jo 

Note that N r (0) is the initial number of class-r jobs in the system, r S 1Z. 

In addition, we define two more derived processes: W(t) = (Wi(t))i e c* an d Y(t) = (Yi{t)) t^c* ■> 
where 

W e (t) := VrN r (t), and Y e (t) := ^(p r t - D r (t)). (17) 

rBl r3i 
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Clearly, W{t) translates the queue-lengths into the (expected) workloads, and Y(t) keeps track 
of the unused capacities at the bottleneck links. (Recall, as we noted in the last section, in a 
fixed-point state, the utility-maximizing allocation at each bottleneck link is A r = p r .) 
The following property of Y{t) will be used later: 

Yi{t) is non-decreasing in t > 0, for all I G L*. (18) 

The above holds because from ()16l) and (|17|). we have 



and 



K{N(s)) <c e = Y^ Pr, for s > 0, 1 G £*. 

To describe the fluid limit and its uniform attraction property, we introduce a sequence of 
networks, indexed by k. Each of the networks is like the one introduced in the last section, but 
may differ in their arrival rates and mean service times (which are also indexed by k). We assume, 
for each r G 1Z, 

\j, — > A r and u£ — > i/ r , as/c — > oo; (19) 
and consequently, — ► p r . In addition, we assume 

^f^^O and ^f^^O, asfc^oo; (20) 

i.e., under the fluid scaling the initial interarrival times and initial residual work requirements are 
negligible. 

We apply the standard fluid scaling to the primitive processes associated with this sequence of 
networks: 

5*(y))) := {^E k {kt), l -S k {ky)y, 
and similarly define the fluid-scaled version of the derived processes: 

(N k {t),D k {t),W k {t),Y k (t)) = (yN k (kt),-D k (kt),yW k (kt),yY k (kt) 
\ ) \k k k k 

When k — > oo and under the assumption (|20|). we know (e.g., [5], Chapter 5) 

E k (t),S k (y)) ^(Xt^y), u.o.c, (21) 



where A := (X r ) r€ Ti and \x := [y r 1 ) r en, and the convergence is uniform on compact sets (u.o.c.) of 
t > 0. 
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The theorem below states that the sequence of derived processes also approaches a limit, the 
fluid limit, which can be characterized as follows. Let N(t) := (N r (t))ren and D(t) := (D r (t)) r en, 
where 

N r (t) := N r (0) + \ r t - v^Drit) > 0, (22) 

D r {t) := I A r {N{s))ds; (23) 
Jo 

and 

A r (n) := ( K{n) °> (24) 

I p r it n r = 0. 

Furthermore, define W(t) := (Wi(t))t££* and Y"(t) := (Yg(i))^ 6 £*, where 

W £ (t) := VrNr{t) and Y £ (t) := ^{p r t - D{t)). (25) 

r3i r3i 

Then, clearly, we have (cf. Q18j)): 

is non-decreasing in t > 0. (26) 
Also note that the processes N(t), D(t), W(t) and are all Lipschitz continuous; and 

H^(t) = W(0) + F(t) (27) 

follows from (|2^1) and (t231). 

Theorem 4 (Fluid limit) Given the utility-maximizing allocation (i.e., A(-) is the solution to 
©), and suppose N k (0) — > iV(0). Then, for any subsequence of these processes, there exists a 
further subsequence, denoted /C, such that, along /C, 

(^(t),^*^),^*^),?*^)) - (A>(i),£(t),iy(i),y(t)) u.o.c. (28) 
with the fluid limit, (N(t),D(t), W(t),Y(t)), following the specification in (J22J), (23), (El) and (1231) . 

The proof of the above theorem is a slight variation of the proof of Proposition 4.2 in Ye et 
al. [HE]) and thus is omitted. We also remark that the convergence of the two primitive processes 
in (|21j) can be used as an alternative to the assumption that these two are renewal processes. 

The next theorem states that when t — > oo, the fluid limit is attracted to a fixed-point state, at 
which the cost (the objective value in the cost minimization problem) is minimal compared with 
any other fluid state with the same workload. 

We start with a definition and a lemma. Denote the cost objective (with A = p) as 

V>(n) := CrK). (29) 
r£TZ 

Below, we shall focus on those differentiable points t of the fluid limit, referred to as "regular" times. 
(Note that the fluid limit is Lipschitz continuous, and hence differentiable almost everywhere.) 
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Lemma 5 Suppose |iV(0)| < B for some B > 0. Then, N(t) is bounded by rj(B), where 77(a) 
(a > 0) is a continuous increasing function with 77(0) = 0; and W(t) is bounded and (component- 
wise) non-decreasing. Consequently, W(t) — > W(oo) as t — > 00. 

Proof. The boundedness of W"(i) follows trivially from the boundedness of N(t): see ()25j) . In 
addition, from (|2l))) and (|27|). we know W(t) is non-decreasing in t. Putting the two together, we 
have W(t) — » W'(oo). Hence, it suffices to the boundedness of iV(i). 
We have, for any regular time t > 0, 

^(iV(t)) = £/V r (i)^(7V r (t)) 



< 



^ ( Pr - A r (N(t))) ■ d 2 U r (N r (t),Pr) 
r£R 

Y, (p r -K(N(t))) ■d 2 U r (N r (t), Pr ) 
rant 

Y, {U r (Nr(t),Pr) ~ U r (N r (t) , A r (7V(t)))) 

ren+ 

Y {U r (N r (t),p r ) - U r (N r (t),A r (N(t)))) < 0, (30) 

ren 



where, the second equality follows from 1)22(1 and (|23|) : the third equality from (|24jl. with 7£^ 
denoting the set of routes with a positive fluid level at time t, i.e., 7^" = {r G 7£ : A^ r (i) > 0}; the 
first inequality is due to concavity of the utility function; and the second inequality is due to the 
fact that A(N r (t)) is the optimal solution to the utility maximization problem in (J2J). (Specifically, 
(A r (N(t)) — p r )r£ii is the ascent direction of the objective function.) Hence, ip(N(t)) < ip(N(0)) 
for all t. From (|29|). we can then conclude that N(t) is also bounded, since C r (n r ), r G 7Z, are all 
strictly increasing and unbounded functions. 
Furthermore, define for any a > 

77(a) = max ip(n) and 

\n\<a 

r } {a) = Y J C r -\fi(a)). 

ren 

As the cost C r is strictly increasing, its inverse used above is well defined and is strictly increasing. 
It is direct to check that the function r/ defined above is increasing with 77(0) = 0. Now, we have 
for all i > 

if>(N(t)) < VW0)) < 77(5), 
which implies (cf. (|29|)). for all r G 1Z, 

C r (N r (t)) < 77(73), 

N r (t) < C-\fj(B)), 
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and finally 



|iV(t)|<^C r 7 1 (^))=^)- 

□ 

Recall, n*(w) denotes the optimal solution to the cost minimization problem in (jlU|) . Given 
a state N(t), the corresponding workload is W(t). If we set uu = W(t) in the cost minimization 
problem, we can write it more explicitly as w(N(t)). Then, the minimizer of the cost objective can 
be expressed as n*(w(N(t)) and the corresponding objective value as ip(n* (w(N(t))). To lighten up 
notation, we shall write this simply as ip*(N(t)). Note the difference between i/;*(N(t)) and tp(N(t)): 
the latter is simply the cost objective evaluated at a give n = (N r (t)) r& Ti, i.e., the objective value 
of a mere feasible solution to (|1U|) ; whereas ip*(N(t)) is the objective value of the optimal solution. 
Both, however, correspond to the same required workload it; = w(N(t). 

Note that since W(t) is non-decreasing in t, as we have shown in the last lemma, and CV's are 
increasing functions, clearly ifj*(N(t)) is also non-decreasing in t. This is in contrast to the non- 
increasing property of tp(N(t)) as established in (J3UJ). Indeed, the key to the proof of the theorem 
below is the following Lyapunov function: 

L(n) = il)(n) - i^*(n), (31) 

where, following the above notation ip*{n) = ip{n*{w{n)). 

Theorem 6 (Uniform attraction) Suppose |iV(0)| < B for some constant B > 0. Then, there 
exist a time, Tb i£ , and an attraction state, N(oo), such that 

\N(t) - N(oo)\ < e for all t > T B , e . (32) 

Furthermore, the attraction state is a fixed point, i.e., N(po) = n*(w), with w = W"(oo) being the 
limit specified in Lemma 13 

Proof. Consider any given regular time t > 0. We first show that for any 5 > 0, there exists a 
a > such that 

\N(t) - n*(W(t))\ > 5 J t L (N(t)) < -v, (33) 

where L(n) is the Lyapunov function defined in (|31|) . Since as argued above, ip(N(t)) is non- 
increasing and ip*(N(t)) non-decreasing, it suffices to show the statement in (|33j) with 4rL(N(t)) 
replaced by ^ip(N(t)). 

From (|3U|) . we know 4iip(N(t)) < 0, with equality holding if and only if N(t) is a fixed point, 
i.e., N(t) = n*(W(t)). This, along with the observation that the function 

f(n) := {Pr — A r (re)) • <9 2 t4( n r> Pr)i where TZ n := {r £ 1Z : n T > 0}, 
is continuous in n (cf. Lemma yields the desired conclusion. 
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The statement in (|33j) implies that for any e > 0, there exists a time T B such that 

\N(t) - n*{W(t))\ < | for all i > T Be . (34) 

On the other hand, the convergence of W(t) —* W(oo) in Lemma along with the continuity of 
the fixed point in Lemma |21 guarantees that 

\n*(W(t)) - n*(W{oo))\ < | for all t > T§ e . (35) 

Hence, the desired result in l|32j) follows from letting Tb^ = max{Tg € , Tg e }, and combining (|34j) 
and □ 

Corollary 7 (a) If N(0) is bounded (N(0) < B), then there exists some T(= Tb )£ ) > 0, such that, 
for any e > 0, the following holds for all t >T: 

\N(t) -n*{W{t))\ < e. 

(b) If N(0) is a fixed point state, then N(t) = N(0) for all t > 0. In particular, if N(0) = 0, then 
N(t) = for all t > 0. 

Proof. Part (a) simply restates what's already proved in (|34[). Part (b) follows directly from the 
property of the Lyapunov function proved above. □ 

Remark. Following TheoremEl when all links are non-bottlenecks, i.e., YlrsiPr < c ^ ^ or an ^ ^ ^> 
we have iV(oo) = 0. (In fact, we have N(t) = 0, t > T, for some T > 0.) This recovers the 
stability result of Ye et al. jSHi- Although the network model in jHH] allows complete processor 
sharing - the link capacities are shared among all jobs present in the network, whereas ours is 
essentially a head-of-the-line processor-sharing mechanism, the two models are equivalent when the 
service requirements follow exponential distributions, which is indeed assumed in |36| . Therefore, 
our results here are more general, in allowing general distributions of the processing requirements 

4 Diffusion Limit and Heavy-Traffic Optimality 

As in Section |21 we introduce a sequence of networks indexed by k. In addition to the limits of the 
arrival rates and mean processing times in ()19|) . we assume the existence of the following limits as 

k — > oo: 

k{\ k r - X r ) -» 6 r>x and k{y k r - v r ) -> 6 r , v , r G K. (36) 

Consequently, we have, 

k{Pr - Pr) = k(X k r u k - X r u r ) -> \ r 6 ryU + v r 6 r ^ := 9 r>p . (37) 

Moreover, we also need to assume the existence of the limits of the standard deviations of the 
inter- arrival times and service requirements: 

and b k r -^b r , r G K. (38) 
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We apply the standard diffusion scaling (along with centering) to the primitive processes, which 
results in the following: 

E k {t) : = i (^E k (k 2 t) - X k k 2 tj , S k (t) := ^ (s k {k 2 y) - {u k )- x k 2 y) . (39) 

Apply the same diffusion scaling to the derived processes: 

N k (t) := lN k (k 2 t), W k {t) := \w k {k 2 t), Y k {t) := ^Y k (k 2 t). (40) 

Next, we re-express the unsealed workload process for the k-th. network as follows: 

WHt) = E>W(*) 
TBI 

= £«*JV*(0) + ([^W " " i S r(D k r(t)) ~ {v k r )- l D k (t))) 

Applying the diffusion scaling to both sides of the above equation, we have 

Wht) := \w k {k 2 t) = X k {t) + Y k {t), (41) 

where 

**(*) : = ^E^(0) + E^(4 fe (i)-^(A fe (0))+E^-Pr)t, (42) 

= ^Ek***-* 2 ^*)]; (43) 

r3i 

and D k (t) := -^jD r (k 2 t) is a variation of the fluid-scaled process D r (t). Note that, Y k (t) in Q43|) 
is consistent with what it was originally defined in (|4U|). It is also completely analogous to its 
fluid-scaled counterpart in 1)25 Jl . Indeed, we also have (cf. (|18[) ). for each k, 

Y k (t) is non-decreasing in t > 0, for all £ E L*. (44) 

Following conventional arguments (the functional central limit theorem, the random time- 
change theorem, etc.; refer to [3], Chapter 5), the processes E k (t), S k (D k (t)) and X k (t), for r £ 1Z, 
converge weakly to some Brownian motion. Specifically, the limit of E k (t), denoted E r (t), is a 
Brownian motion with zero mean and variance \fa 2 . Assume, for ease of exposition, N k (0) = for 
all r € 1Z and all k. (More general initial conditions can be handled by addressing the convergence 
of the sequence of initial states, and the associated inter-arrival times and service requirements 
of the initial jobs. Details are similar to those in [22].) Then, D k (t) — > p r t. Hence, S k (D k (t)) 
converges to S k (p r t), which is a Brownian motion with zero mean and variance v~ - 3 b 2 ; and X k (t) 
converges to the following limit: 

X e {t) := E e r,pt + E v r (M*) ~ , (45) 

rBi r3l 
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which is a Brownian motion with drift YlrBt @r,p an d variance J^r^ei^r ^r a r + y r 

For the derived processes, denote their limits (to be proved in the next theorem under a single- 
bottleneck condition) as follows: 

N(t) = (Nr(t)Un, W(t) = (W t {t)) ieC * , Y(t) = (Y e (t))eec* ■ 
These processes are characterized by the following relations, for all t > 0, £ £ C* and rGK: 

W i (t)=X i (t)+Y e (t)>0; (46) 
Yi(t) is non-decreasing ini; with 5^(0) = 0; (47) 

W e (t)dY e (t) = 0; (48) 
N r (t) = n*(W(t)). for t > 0, r G K. (49) 



o 



It is known (e.g., 5 , Chapter 6) that given X(t) the relations in ()46l) . (|47|) and (|48() . which constitute 
the so-called Skorohod problem, uniquely define the processes Y{t) and W{t): Y = &(X) and 
W = *$>(X), with <£(•) and ^(-) being Lipschitz continuous mappings. In particular, when X(t) 
is a Brownian motion, W(t) is a reflected Brownian motion (RBM), and Y{t) is the associated 
regulator. 

Theorem 8 Suppose the heavy-traffic condition in (J^J holds, with a single bottleneck link, i.e., 
L* = {£*} is a singleton set. Under the utility-maximizing allocation, we have the following results. 

(a) (Diffusion Limit) The following weak convergence holds when k — > oo: 

(w k (t),Y k (t),N k (t))) (w(t),Y(t),N(t)) , 

with the limit following the specifications in ()46|) through ()49|) : in particular, W is a single- 
dimensional RBM, Y is the associated regulator, and N is the fixed-point to the cost mini- 
mization problem with w = W . 

(b) (Asymptotic Optimality) W and N are minimal in the following sense: Let W k,G and 
jyk,G dgnotg t ne processes associated with any feasible allocation scheme G. Then, for all 
t > 0, we have 

liminf E[W k > G (t)} > E[W(t)} and liminf V E[C r (N k ' G {t))} > V E[C r (iV r (t))]. (50) 

fc— >oo fc— >oo ^— ^— 

reft reft 



The proof of the above theorem is deferred to Section 14.31 Below, we first motivate the single- 
bottleneck condition, a key requirement in the theorem. 

4.1 The Single-Bottleneck Condition 

Here we show that the single-bottleneck condition in Theorem |S] is equivalent to the so-called 
resource pooling condition, which has been widely used in related studies on the heavy-traffic 
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limits in various stochastic processing network; refer to, e.g., ^1 OH El HOI H31 HB EDI l^lj - 
among others. 

Following Harrison ^2]) the resource pooling condition can be stated as the uniqueness of the 
dual optimal solution to what's known as a "static planning LP" (linear program), which in our 
context takes the following form. First, the primal LP, with £ and (A r ) rg -/j as decision variables: 

(P) max £ 

s.t. -A r + p r £ < 0, r G TZ; 

£ > 0, A r > 0, r G n. 
Next, the dual, with p r , r £ 1Z, and 7T£, £ G £, as variables: 

(D) min Q7r^ 

S.t. p r p r > 1; 

— p r + 7T^ > 0, r eTZ; 

p r > 0, r £ 1Z; ire > 0, £ £ £. 

Here, the dual variables 7r^, £ £ £, are the shadow prices of the link capacities; andp r is the marginal 
cost for processing a class-r job, r G 1Z. The resource pooling condition can then be stated as the 
uniqueness of the (p r )ren part of the dual optimal solution. (No requirement on the tt part.) 
For our LP presented above, we claim the primal optimal solution is: 

£ = 1; A r =p r , r G K; (51) 

whereas the dual optimal solution is characterized as follows: 

Pr = ^2^e= ^2 n t> r£K; (52) 

and 

J2c m = l; ir e = 0, i^C. (53) 

To justify, it is straightforward to check that the above satisfies: (a) primal feasibility, (b) dual 
feasibility, and (c) complimentary slackness. Indeed, the only less-than trivial part is to check the 
dual constraint, ^2 r& -nPrPr > 1. It follows from 

p r p r = p r TT £ = TT t ^ p r = 7T^ ^ p r = ^ 7T£Q = 1, 

where the second last equality follows from Y^r^ePr = c t f° r ^ £ <£*• 
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From Q52JI and (|53|) . it is clear that when there is a single bottleneck link, i.e., C* = {£*}, then 
the following dual optimal solution is unique: 

tt£* = cj}; tt£ = 0, £ / T; 
p r = ne* = c^} , r3f; p r = 0, r$£*. 

Otherwise, it is not, as we can choose different (iri)i£C* values to satisfy the first equation in (|53|) . 
and thereby lead to different p r values following (|52j). 

As pointed out by Stolyar (Theorem 3 of [SOI)) the uniqueness of the optimal dual solution is 
equivalent to the uniqueness of a vector (p r )r£ii that satisfies: 

EPrPr = max > p r A r . 
AeM^ 

reR reR 

Clearly, p := (p r )ren ls the normal vector to the outer boundary of M at p. Since the outer 
boundary is formed by a set of hyperplanes, each of which corresponds to a link, p is unique if 
and only if it lies in the interior of a face of M. This provides a geometric interpretation of the 
single-bottleneck condition. 

We conclude this part by the following lemma, which says that when the network state n is 
close to a fixed point, the capacity of the single bottleneck link will be fully utilized. The proof of 
the lemma is postponed to the Appendix; the lemma itself will be used in the next subsection. 

Lemma 9 Suppose I* is the only bottleneck link in the network. Then, for any given constant 
e > 0, there exists another constant a = cr(e) > 0, such that, for any state n with workload 
w i* = 52r3t* u mr satisfying 

\n — n*(w)\ < a, and wp* > e; 
we will have ^2 r3 g* A r (n) = q*. 

4.2 A Key Lemma 

To simplify notation, since there is a single bottleneck link £*, we will drop the subscript £* from 
-Xji(i), Y e k *(t), and W^(t), and from their (proposed) limits, X e *(t), Ye*(t), and Wp(t). For the 
rest of this subsection and the next one, these are all scalar processes. In addition, we find it more 
convenient to adopt a sample-path approach based on the Skorohod representation theorem, i.e., 
to turn the weak convergence into a probability one convergence (u.o.c.) of suitable copies of the 
processes on a common probability space. Indeed, in the rest of this section and the next section, 
we shall focus on a given sample path. 

We shall focus on the time interval [r, r + 5], where r > and 5 > 0. Let T > be a fixed time 
of a certain magnitude to be specified later. Let the index k be a large integer. Divide the time 
interval [r, r + 5] into equal segments of length T/k, a total of kS/T such segments. (Without loss 
of generality, assume S/T is an integer, and hence, so is kS/T.) Then, for any t 6 [r, r + S], we can 
write it as t = r + (jT + u)/k for some j = 0, • • • , k5/T and u G [0, T\. Therefore, we write 

W k {t) = W k (r + J -^-^) = \w k {k(kr + jT + u)) := W k ' j (u), uG[0,T], j<^. (54) 
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X(t) 




That is, for each time point t, we will study the behavior of W k (t) through the fluid process, W ^(u), 
over the time interval u € [0, T]. Similarly define N ^(u) and Y k ^(u) as the fluid "magnifiers" of 
N k (t) and Y k (t). 

The above rescaling of W k (t) is illustrated as an example in Figure |21 This rescaling technique 
enables one to investigate the structure of diffusion scaled processes (e.g., W k (t)) using the available 
results concerning the fluid scaled processes (e.g., W k {t)). Such a technique appeared in various 
forms in El El El ESI ED] 



Diffusion 
scaling 



W\t), ?g [z,z+d] 



T+S 



Fluid 
scaling 



q T 2T j T q +1 ) T [k5/T]T 
-i 1 — i 1 — i 



W%), MG [0,T] 



Figure 3: Rescaling of processes 



Lemma 10 Consider the time interval [r, r + 8], with r > and 5 > 0; pick a constant C > such 
that 



sup \X{t') - X(t")\ <C; 

t',t"e[r,T+8] 



(55) 
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and suppose 

lim W k {r) = X , and lim N k {r) = N*(x), ( 56 ) 

fc— >oo k^oo 

for some constant \ > 0. Let e > be any given (small) number. Then, there exists a sufficiently 
large T such that, for sufficiently large k, the following results hold for all non- negative integers 
j < k5/T: 

(a) (uniform attraction) 

\N k ' j (u) - n*{W k > j {u))\ < e, for all u G [0,T]; 

(b) (boundedness) 

W k ' j (u) <X + C + 0(e), for all u G [0,T]; 
where lim e ^oO(e) = 0; i.e., W k 'i{u) is uniformly bounded; and hence, so is N k ^{u); 

(c) (complementarity) if W '•'(u) > e for all u G [0, T], then 

y fc ' j (it) - Y k 'i (0) = 0, for all u G [0, T] . 

The proof of the lemma is postponed to the appendix. Below, we first comment on the key 
points of the lemma, followed by proving Theorem |S] in the next subsection. 

Remarks. 

• The main difficulty in proving Theorem |S] is the lack of complementarity between W k (t) and 
Y k (t). Specifically, while (jUJ) and (jSJ) hold for W k {t) and Y k {t) as counterparts to (QU) 
and (|47|) for the limiting processes Wit) and Y(t), the complementarity in (|48|) does not 
necessarily hold for W k [t) and Y k (t). Indeed, for an arbitrarily given k, it may not hold 
when W k (t) is positive but N k (t) deviates significantly from the fixed point state. Part (c) 
of the above lemma asserts, however, that complementarity will hold when k is large enough. 

• Recall from Theorem El we know that the fluid-scaled queue-length process will approach 
the fixed-point after a certain time of length T. This means, without the fluid scaling, the 
time to reach the fixed-point is in the order of kT, which translates into a time length in the 
order of T/k for the diffusion-scaled process N k {t). In this sense, N k (t) approaches the fixed 
point almost "instantaneously". This is confirmed in part (a) of the above lemma, i.e., N k (t) 
evolves closely around the fixed-point process (for sufficiently large k). As the fixed point is 
unique given the workload w, the process N k (t) can be characterized (approximately) by the 
workload process W k (t). 

• Furthermore, Lemma guarantees that, under the single-bottleneck condition, in the fixed- 
point state, there will be no unused capacity at the bottleneck link, as long as there is some 
positive workload (no matter how small) associated with the fixed-point state. Hence, Y k (t) 
will not increase, which is the required complementarity. 
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4.3 Proof of Theorem HI 

As before, we focus on a fixed sample path. Since Y k (t), for each k, is a process that is nondecreasing 
and right continuous with left limit (RCLL), we are guaranteed that for any subsequence of these 
processes there exists a further subsequence, denoted /C, that converges to a limit Y(t), which is also 
nondecreasing and RCLL; note that Y(t) is continuous for almost all time t and that at the moment 
this convergence is guaranteed only for those time t at which Y(t) is continuous. Consequently, we 
have, W k (t) -> W(t), along the same convergent subsequence /C, with W(t) satisfying the relation 
in 1)46)1 : W(t) = X{t) + Y(t). Furthermore, W(t) is bounded, following part (b) of Lemma |lfll 
(We can choose r = and any 5 in the lemma; and hence, x = 0-) This implies that Y(t) is also 
bounded, since X(t) is bounded by C (cf. 1)55)) ). 

Based on part (a) of Lemma ITUl we have, also along the subsequence /C, N k (t) — > iV(i) := 
n*(W"(t)). 

Next, we argue that the limit, Y(t) is, in fact, continuous; hence, so are W(t) (since X(t) is 
continuous) and N(t) (since n*(w) is continuous according to Lemma OJ). That is, the convergence 
of Y (t), W k (t) and N k (t) to their limits holds for all time t, not just for the time points at which 
they are continuous as argued above. 

Use contradiction. Suppose Y{t) is discontinuous at tj, specifically, 

Cj := Y(tj+) - Y(tj-) > 0. (57) 

Pick the time interval [r, r + S] to include t j; where r is a continuous time of Y(t) and close enough 
to tj and 5 is small enough, such that (i) the inequality in l(55)l holds with C < Cj/2 and (ii) 
|VF(t) — W(tj—)\ < C. (This is possible since X{t) is continuous and W(t) has left limit at any 
time t.) Based on (i), we can invoke part (b) of Lemma ITO) letting e — ► 0, and letting W(t) = x 
be the limit of W k (r) along the subsequence JC. This, along with (ii), yields 

W(t) < x + C = W{t) + C < W{tj-) + 2C. 

From 1)57)) and the continuity of X, we have 

W(tj-) = X{tj-) + Y(tj-) = X(tj+) + Y(tj+) -Cj = W(tj+) - Cj. 

Combining the above, we have 

W(t) < W(tj-) + 2C 

= (W(tj-) + Cj)-(Cj-2C) 
= W(tj+)-(Cj-2C). 

That is, 

W(t j+) - W(t) >Cj-2C>0 

for any t E [t,t + 5], contradicting the right continuity of W (at t = tj). 

We can now make use of part (c) of Lemma ITUl to claim the complementarity in 1)48)) . i.e., if 
W{t) > for some time t > 0, then there exist a small time interval (r, r + 5) 3 t such that Y(t) 
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does not increase, i.e., Y(s) — Y(t) = for all s G (r, r + 5). The discussion of the case t = is 
similar and hence omitted. 

Having proved that the convergence, along the subsequence /C, to the limit (Y(t),W(t), N(t)) 
holds for all t, and that the limit is continuous and satisfies all the requirements in (j46|l - (|49J) . we 
can invoke the uniqueness of the solution to the Skorohod problem (e.g., [3], Chapter 6) to conclude 
that the convergence holds for the original (full) sequence u.o.c. 

Finally, we prove the optimality in part (b) of Theorem |HJ Let 

Y G {t) := liminf y fe ' G (i), and W G (t) := liminf W k ' G (t); 

fc— >oo fc— >oo 

and note that X ' (t) — ► X(t) is independent of the resource allocation scheme G. The following 
is then directly verified: 

W G (t) = X(t) + Y G {t) > for all t > 0, 
Y G (t) is non-decreasing with Y G (0) > 0; 

whereas the complementarity need not hold for (W G (t),Y (t)). Hence, we have 

liminf W k ' G (t) > W(t), 

k^oo 

following the well known minimality of the Skorohod problem. This inequality then implies 



liminf V C r (N k ' G (t)) > V C r (N r (t)), 



fc— >0O 



since N(t) is a fixed-point (see (|49j0 and hence a minimizer of the cost function. The above two 
inequalities then imply the two inequalities in ()5UI) . □ 



5 Appendix 

5.1 Proof of Lemma O 

Let 1Z* := {r G 1Z : I* Er} denote the set of routes that involve the bottleneck link £*. Partition 
the link set £ into C = {£*} U £ U £", where £' := {I / £*, £ G r for some r £ 11*} denotes the 
set of links, excluding the bottleneck link £*, that are used by at least one route r G 1Z* , and £" 
collects all other links. 

We prove the lemma by contradiction. Suppose the lemma does not hold. Then, there is a 
sequence of states h l = {h l r > 0, r G 1Z} (i = 1, 2, ...) satisfying 

\n l — n*{w % )\ — > as i — > oo, u> 1 = f r n* > e, for all i = 1, 2, • • • , 

such that 

y A r (n l ) < q* for all i = 1,2, — . 
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We scale the sequence of states by letting n % = 4^h % , so that w % = Ylr^e* = e -> f° r alH = 1, 2, . . .. 
Then, from the above and the homogeneity property in (Jjy), we have 

n % -> n*(e) as i -> oo; E A r (n l ) < q. for all i = 1, 2, • • • . (58) 

Rewrite the utility optimization problem as follows: 

max U r (n r , A r ) + U r (n r ,A r ) 

s.t. A r < Q* 

E A r + ^ Ar < Q for £ G £', 

r£fc*:r3£ r£ll\n*:rBe 

E A r < c £ for £ G 

reft\ft*:r3£ 

A r > for r G ft. 

As the allocation A(n l ) is the optimal solution to the above, there exists a set of Lagrange multiplies 
{9g,£ G £} such that, by way of the KKT condition and the inequality in (|58|). the following holds 
for alii = 1, 2, ■ • •, 

6\ > for ^ G £, 
ej. = 0, 

d 2 U r (n*.,A r (n i ))<J2e i l = E °e for r G 7£*, (59) 

^E/rK-M™')) <E * = E 0j + E 9 i forr£K\R*. (60) 

ter tern£' eemc" 

First, summing up all the inequalities in (|59|) yields 

E d 2 c/ r «,A r K)) < E E ^ 

reft* reft* iemC 

£e£' reft* 

where |7£| denotes the total number of routes in the set 1Z. Hence, we have 

liminf V0j > lim — V d 2 U r {n i r , A r (n 1 )) 

i— >oo ^— ' i— »oo 7v ^ — ' 

£e£' 1 1 reft* 

I, E d 2 U r (n*(e), Pr )>0, (61) 



1^1 



where the equality follows from the continuity of the allocation A(-) (LemmaEJ, property (i) in 
Proposition^! and the continuity of c?2 f/ r ( • , •) (recall U r is assumed to be twice-differentiable) . 
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Next, choose a > and > 0, sufficiently small, such that 

p T + a + \K\K*\(3 <a ioiallleC'. 

For each index i, denote C'(i) := {£ G C : 91 > 0}. From the continuity of the allocation, we know 
A r (n' 1 ) — > A r (n*(e)) = p r for all r G 7?*. Thus, for sufficiently large i, we have 

^2 Kin 1 ) < Yl Pr+a for i G 

Since 0) > for any £ G C'(i), we have, according to the KKT condition, 

^A r (n i ) = c £ for£e£'{i). 

Consequently, for sufficiently large i, we have, for each £ G 

A r (n J ) = q — £ A r (n*) 



> c e 



\r<=Tl*:r3e ) 



As the number of terms in the first summation is at most \R, \ 7£*|, there exists at least one route, 
denoted as r(£) G TZ \ TZ* (with £ £ C'(i)), such that 

A rW (n i )>/3. (62) 

According to the KKT condition, any inequality in IjOUj) that corresponds to a route r{£), £ G 
should hold as equality. Summing up all such equalities yields 

£ 5 2 ^ w « w ,A rW K))= ^ E ^ E ^ 

£e£'(i) \jGr(£)n£' jer(e)nc" 

* E E ^ E = ( 63 ) 

£G£'(i) j£r(£)nC> £e£'(i) 

where the second inequality is due to the fact that the summation Yljer(£)nC @j contains the term 
9\, and the last equality follows from the definition of the set C'(i). 
On the other hand, in view of (|62j) . we have 

Y d2U r{e) (n l r{e) ,A r{e) (n 1 )) < £ d 2 U r{e) {nl, {e) , (3) 
teC'ii) eec(i) 

^ E E d 2 U r (4,/3)<\£'\ Y d 2U r (K,P) 

eeO(i) r&1l\1l* r£TZ\1Z* 

^ as i -> oo. (64) 

(Note that, should the set C'(i) be empty for some index i, the above still holds by default.) Putting 
together (|6T?|) and we have Yleec ®t ~ > 0> as z — > oo, contradicting the limit in (|HT|) . □ 
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5.2 Proof of Lemma 1101 
Preparations 

Let bN and b\y be any constants such that the following inequalities hold, 

w < bjsr\n*(w)\, \n*(w)\ < bww. 

In fact, since 

( min v r | n* <w = ^ v r n* < I max v r j n*, 

V r ^ / r -^» r9 ^» \ r9 / rSrf , 

the two constants can be chosen as 



&7V = max v T , bw = I min 



-1 



Next, define the following constants, 

Bn,i = r](bwe + e) + e, %,i = b N B N ^ + e; 
Svk,2 = max{Sty,i, x + e} + (C + e), 5jv,2 = byyBw,2 + e; 

where the function ry(-) appeared in LemmaEJ and all other quantities are specified in the statement 
of the lemma under proof. The constants defined above will be used to bound processes N k (t) and 
W k (t) for t G [t, r + 6] and sufficiently large k. 

Next, we specify the time length T (stated in the lemma under proof) as follows: 

T > max{T fewe+£)e , T max { B ^ x B ^ 2 } e / 2 ; ^max{_B J v i i,B J v, 2 },o-/2}> (65) 

where the terms on the right hand side were used in the proof of Theorem El and a = a(e) is 
specified in Lemma (with e given in the lemma under proof). Note that T is long enough so 
that in the fluid network, the fluid state N(t) will be close enough (by an error bound of e) to the 
fixed-point state, from an initial state iV(0) that is bounded by ftyi/e + e; or close enough by an error 
bound of e/2 or cr/2), from an initial state that is bounded by max{£>7v,i> -f?/v,2}- 

With the quantities defined or specified above, we state what we want to prove, in terms of 
parts (b) and (c) of the lemma, in the following stronger form (part (a) remains the same): For 
sufficiently large k, the following results hold for all non-negative integers j < k5/T: 

(a) \N k >i(u) - n*(W k >i(u))\ < e, for all u G [0,T]; 

(bl) if W k ' j (u) < e(< C) for some u G [0,T], then, for all u G [0,T], 

W k ' j (u) < B Wi i, \N k ' j {u)\ < B NX , (66) 

(b2) if W k ' j (u) > e for all u G [0,T], then, for all u G [0,T], 

W k ' j (u) < B W ,2 (=X + C + 0(e)) , \N k > j (u)\ < B N>2 , (67) 

and 

Y k ' j (u) -Y k ' j {0) =0. 
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Step 1 of the Proof 

Here we prove the three parts of the lemma, (a, bl, b2), for j = 0. Note that by way of the 
construction, we have 

(W k '°(0),N k '°(0)) = (W k (T),N k (T)), 

and hence, 

(W k >°(0),N k >°(0)) - (x,n*( X )), as k - oo, 

following ()56j) . Then, from Theorem 0] and Corollary [7| (with W(0) and iV(0) replaced by x and 
n*(x) respectively), we have, as k — » oo, 

iV fc '°(u)) -> (W{u),N{u)) u.o.c. of « € [0, T], 

with (W",iV) satisfying (W(u),N(u)) = {Xi n *{x))i f° r w > 0. (Note that the convergence here is 
along the whole sequence of k rather than a subsequence since the limit is unique.) Since n*(w) 
is continuous in w (Lemma |3J), there exists a' > such that, for any feasible workload level w' 
satisfying \w' — w\ < a', we have, 

\n*{w')-n*{w)\ < e/2. 

Let k be sufficiently large such that 

\W k '°(u) -x\ < a*, and \N k '°(u) - n*(x)\ < e/2, 
for all u G [0, T]. Then, combining the above, we have, 

\n*(W k >°(u)) - n*( X )\ < \ for all u G [0,T], 

and furthermore, 

\N k '°{u) - n*{W k '°(u))\ < \N k '°{u) - n*{ X )\ + \n*(W k '°{u)) - n*{ X )\ < e, 

for all u G [0, T]. That is, (a) holds when j = for sufficiently large A;. 

We now verify (bl, b2). First, from the established result in (a), we know that W k '°(u) is 
arbitrarily close to x f° r all u G [0, T] when k is sufficiently large. This fact directly leads to the 
inequalities in (bl,b2) for j = 0. Next, recall that the number a = a(e) is chosen from Lemma El 
with e given in Lemma ITUl For sufficiently large k, we have, for all u G [0, T], 

\N k >°(u) - n*( X )\ < |, \n*( X ) ~ n*(W k >°(u))\ < |, 

following what's already established above. Putting the two together yields 

\N k '°(u) -n*{W k '°(u))\ < a. 
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Finally, we have, for any u £ [0, T], 

yW( u ) - Y k '°(0) = /V (p r - A r (N k '°(s))) ds 

Jo r9 ,, v ' 

c e *-Y,MN k >°(s)))ds = 0, 

where the first equality follows from the definitions of the processes Y k, i(u) and Y k (t), along with 
Q16|) . and the fact that N k,0 (s) > for all routes r 3 £* and all time s € [0, T] under the condition 
in (a); the second equality follows from the heavy-traffic condition in (jUJ); and the last equality from 
Lemma El Thus, we have shown the complementarity in (|68|) of (b2), for j = 0. 

Step 2 of the Proof 

We now extend the above to j = 1, . . . , k5/T. Suppose, to the contrary, there exists a subsequence 
K,\ of k such that, for any k E K\, at least one of the properties (a, bl, b2) does not hold for 
some integers j £ [1, k5/T]. Consequently, for any k £ K\, there exists a smallest integer, denoted 
as jk, in the interval [l,kS/T] such that at least one of the properties (a, bl, b2) does not hold. 
To reach a contradiction, it suffices to construct an infinite subsequence fC' 2 C JC%, such that the 
desired properties in (a, bl, b2) hold for j = jk for sufficiently large k £ /C^. To construct such a 
sequence, we will first find a subsequence IC2 C /Ci such that the property (a) holds for j = for 
sufficiently large k £ /C2. Next, we partition /C2 into two further subsequences, IC2 = IC3 U /C4; and 
show that the conclusion of (bl) holds for sufficiently large k £ IC' 3 C IC3, and that the conclusion 
of (b2) holds for sufficiently large k £ /C4. Finally, the subsequence K,' 2 = IC' 3 U /C4 is what we need. 

From the proof in Step 1, under what's assumed above, properties (a, bl, b2) hold for j = 
0, jk — 1, k £ JC\. Specifically, for J = Jfc — 1, we have 

\N k ' jk - 1 {0)\ < max{B Njl ,B N;2 }, for all k £ K\. 

Therefore, the sequence {N k, ^ k ~ 1 (0), k £ /Ci} has a convergent subsequence. Then, by Theorem El 
there exists a further subsequence IC2 C JC± such that 

(W k ' jk ~ 1 {u),N k ' jk - 1 (u)) -> (W(u),N(u)) u.o.c. of u > 0, as fc -> 00 along /C 2 , (69) 

with |iV(0)| < max{Sjv,i, -Bjv,2}- Then, we have 

|iV fe ' J ' fc - 1 (u) -n*(W ,fcj '*- 1 («))| 
< liV^- 1 ^) - N(u)\ + |JV(u) - n*(W"(«))| + \n*(W(u)) - n*(W" fcJ * -1 («))| 
— ► |iV(u) — n*(W^(n))| u.o.c. of n > 0, as A; — > 00 along K.2- 

Moreover, since T > r max r Bjv lt B N2 },e/2 an d taking into account Corollary [7[ we have 

\N(u) - n*{W(u))\ < I for all u>T. 
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Therefore, for sufficiently large k € /C2, we have, for u € [0, T], 

\N k ' jk {u) -n*(W k > jk {u))\ = \N k ^- l {T + u)-n*{W Kjk - l {T + u))\ < e. 

That is, (a) holds with j = for sufficiently large k S IC2 (C /Ci). 

Next, we partition /C2 into /C3 U /C4 according to the conditions given in (bl,b2), i.e., 

£ 3 = {A; € £ 2 : W* J *(u) < e for some u G [0, T}}, 
K A = {keK 2 : W k > jk (u) > e for all u £ [0,T]}. 

Note that at least one of the two sequences /C3 and /C4 must be infinite. 

Suppose /C3 is infinite. Then, for each k 6 £3, there exists a fixed G [0,T] satisfying 

VF fe ' ifc K) < e. (70) 

Furthermore, we can choose a subsequence /C 3 C /C3 such that, for some u' € [0, T], 

Uk —> v! as — > 00 along /C 3 . 

Note that the convergence in is valid for the subsequence IC' 3 C /C2 too. Before proceeding, we 
refine the bound for the initial state N(0). First, note that 

W(0) < W(u') = lim W k ' jk (u k ) < e, 

fc^oo 

where the first inequality follows from the increasing property of W(t); and the second one from 
1)70(1 . Next, we have the following estimation, 

\N(0) -n*(W(Q))\ 

< \N(0) - N k ' jk (0)\ + \N k ' jk (0) - n*{W k ' jk (0))\ + \n*{W k ' jk (0)) - n*(W(0))\. 

The first and the third terms on the right hand side above will approach as k — > 00; whereas the 
second term is bounded by e for sufficiently large k G fC' 3 , since property (a) holds for k E /C 3 C /C2. 
Hence, 

\N(0)-n*(W(0))\ < e; 
from which we obtain a refined bound for N(0): 

\N{0)\ < \n*(W(0))\ +e<b w e + e. 

As a by-product, we have 

\N(u)\ < r](\N(0)\) < r](b w e + e), 
where the first inequality follows from Lemma [5J and, letting u — > 00, 

\n*(W(oo))\ = \N(oo)\ < 7]{b w e + e) < B Njl , 
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where the equality follows from the fact that iV(oo) is a fixed point state (cf. Theorem 0) . 
Now, for sufficiently large k G JC' 3 , we have, for all u G [0,T], 

\N k ' jk (u)\ < \N(u)\ + e<ri(b w e + e) + € = B Njl , (71) 
W k ' jk (u) < W(u) + e < W(oo) + e 

< b N \n*{W{oo))\ + e<b N B N , 1 + e = B WA , (72) 

where the first inequality in (|71|) follows from (|69|) . and so is the first inequality in (|72|) ; while 
the second inequality in 1)72(1 follows from the increasing property of the workload W. The two 
inequalities in (|71|) and (|72j) together imply that (bl) holds for j = jj. for sufficiently large k G IC' 3 . 

Next, suppose /C4 is infinite. The convergence in is valid for the subsequence /C4 C /C2 too. 
Recall, a = er(e) is chosen from LemmalHI Then, given the time T > T max { Bjv ljBjv 2 } !Cr /2> we have, 
for all u G [0,T], 

|JV'(T + ti)-n*(W'(r + «))| < |, 

due to Corollary EJ In addition, as k — > 00 along the sequence /C4 (c /C2), we have the following 
convergence (u.o.c. of u > 0), 

|jV*dV-l( u ) _ iV(it)| 0, and |n*(IU(u)) - ^(W"*^*- 1 ^))! -» 0. 

Therefore, for sufficiently large k G /C4, we have the following bound for all it € [0, T], 

\N k ^~ l (T + u)- n^W^-^T + u))\ 

< \N k > jk ~ l (T + u) - N(T + u)\ + \N(T + u) - n* (W (T + u))\ 
+ \n*(W(T + u))-n*(W k ' jk - 1 (T + u))\ 

< cr. 

From Lemma El and the fact that W^' 3 '* (ti) > e for all u G [0, T], we have 

F Jrl (T + u) does not increase in u G [0,T], 

or equivalently, 

^ _ y fc,j fc ^ = Q for all n e [ 0) T] , (73) 

for sufficiently large k G /C4. 

Using the complementarity property just established, we estimate the upper bounds for 
and N k ^ k (u), for u G [0,T]. For a given (sufficiently large) k G /C4, there are two mutually exclusive 
cases: 1) the condition (as well as the conclusions) in (b2) holds for all j = 0, ■■■■,3k) 2) the condition 
in (bl) holds for some j = 0<j<jk — 1. 

In the first case, the process Y k ^{u) does not increase in u G [0,T], for j = 0, ...,jk — 1. Thus, 
we have, for sufficiently large G /C4, 

iy fc .^( u ) = iu fc ' (o) + 2 (iy fcJ '(T) - iu feJ '(o)) + (w k ' jk (u) - w k > jk (ofj 

j=0 
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= W kfi {$) + (x k j{T) - X fcj '(0)) + (x k '^{u) 

3=0 
Jfc-1 

+ (Y k ' j (T) -r^'(0)J + (Y k ^(u) 

3=0 

3k~ 1 

= VF fc '°(0) + (x fe ' J '(T) - X k ' j (0^j + (x fe '^(u) 
= W k {T) + {x k (T + j k T/k + u/k)-X k {r)) 

< ( X + e ) + (C + 6), 

where the inequality follows from and (|56[). 

Under Case 2, let j° be the largest integer such that the condition in (bl) holds. Thus, for all 
j=jfc + l<j< jk, the condition and results in (b2) hold, and hence Y k ^{u) does not increase in 
u £ [0, T}. Then, similar to Case 1, we have, for sufficiently large k £ /C4, 

W k ' jk (u) = W k ' j °(T) + (W k ' j (T) - W k > j (0^j + f W*J*(u) - W k ' jk (0^j 

= iT^(T) + (± fc (r + j fc T/A; + «/*) - X fc (r + j^T/k + T/k)) 
< B w>1 + (C + e). 

where the inequality is due to the bound (|66jl in (bl) with j = j9 and the definition of the constant 
C in (|55|> . Then, synthesizing the bounds in the two cases, we have, for sufficiently large k € JC4 
and for all u G [0, T], 

W*J*(u) < max{( X + e) + (C + e), + (C + e)} = B wX , 

and furthermore 

\N k ' j Hu)\ < \n*(W k ' jk (u))\ + e 

< 6 w PF fc ' ife (u) + e < b w B Wt2 + e = Sjv, 2 - 

The above two bounds, together with the complementarity property in Q73|). imply that (b2) holds 
with j = jfc for sufficiently large k G /C4 . 

Finally, let /C2 = /C3 U /C4 (c /C2 C /Ci). Then, the properties in (a, bl, b2) with j = jk hold 
for sufficiently large k E JC' 2 (C /Ci). □ 
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